Oliver Castle was a medieval tower house, located in upper Tweedsdale in the Scottish Borders. The site of the hillfort known as Oliver Castle is to the north of the village of Tweedsmuir, although the site of the tower house is less certain. Mentioned in a document of c.1200, it was originally part of the line of peel towers along the Tweed Valley. It was replaced in the seventeenth century by a house, which was itself replaced in the late 18th century by the present Oliver House. For most of its existence the property has been owned by members of the Tweedie family.